Collaborative human-autonomy semantic sensing through structured POMDP planning

Autonomous unmanned systems and robots must be able to actively leverage all available information sources — including imprecise but readily semantic observations provided by human collaborators. This work develops validates a novel active collaborative human–machine sensing solution for robotic gathering optimal decision making problems, with an example implementation of dynamic target search scenario. Our approach uses continuous partially observable Markov process (CPOMDP) planning generate vehicle trajectories that optimally exploit imperfect detection data from onboard sensors, as well natural language can specifically requested sensors. The key innovations are method the inclusion querying/sensing model in CPOMDP based autonomous process, scalable hierarchical Gaussian mixture formulation efficiently solving CPOMDPs state spaces. Unlike previous state-of-the-art approaches this allows large, complex, highly segmented environments. is demonstrated validated real human–robot team engaged indoor capture scenarios on custom testbed.